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Abstract. Numerical causal derivative estimators from noisy data are 
essential for real time applications especially for control applications or 
fluid simulation so as to address the new paradigms in solid modeling and 
video compression. By using an analytical point of view due to Lanczos 
[9] to this causal case, we revisit n th order derivative estimators origi- 
nally introduced within an algebraic framework by Mboup, Fliess and 
Join in [14115] . Thanks to a given noise level 5 and a well-suitable inte- 
gration length window, we show that the derivative estimator error can 

q+l 

be 0(8 n+i+s) where q is the order of truncation of the Jacobi polyno- 
mial series expansion used. This so obtained bound helps us to choose 
the values of our parameter estimators. We show the efficiency of our 
method on some examples. 

Keywords: Numerical differentiation, Ill-posed problems, Jacobi orthog- 
onal series 

1 Introduction 

There exists a large class of numerical derivative estimators which were intro- 
duced according to different scopes ( |8I18I11 22 20 16 17\). When the initial dis- 
crete data are corrupted by a noise, numerical differentiation becomes an ill- 
posed problem. By using an algebraic method inspired by [6113110] . Mboup, 
Fliess and Join introduced in |14ll5j real-time numerical differentiation by inte- 
gration estimators that provide an effective response to this problem. Concerning 
the robustness of this method, |4I5] give more theoretical foundations. These es- 
timators extend those introduced by |9ll9l23j in the sense that they use Jacobi 
polynomials. In [Tl], the authors show that the mismodelling due to the trun- 
cation of the Jacobi expansion can be improved by allowing a small time-delay 
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in the derivative estimation. This time-delay is obtained as the product of the 
length of the integration window by the smallest root of the first Jacobi polyno- 
mial in the remainder of series expansion. 

In we extend to the real domain the parameter values of these Jacobi 
estimators. This allows us to decrease the value of this smallest root and conse- 
quently the time-delay estimation. In |12j , we study for center derivative Jacobi 
estimators the convergence rate of these estimators. 

Thanks to these results, we propose in this article to tackle the causal con- 
vergence rate case. We give an optimal convergence rate of these estimators 
depending on the derivative order, the noise level of the data and the trunca- 
tion order. Moreover, we show that the estimators for the n th order derivative 
of a smooth function can be obtained by taking n derivations to the zero-order 
estimator of the function. Hence, we can give a simple expression for these esti- 
mators, which is much easier to calculate than the one given in [12] . 

This paper is organized as follows: in Section [5] the causal estimators in- 
troduced in [14j are studied with extended parameters. The convergence rate of 
these estimators are then studied. Finally, numerical tests are given in Section [3] 
They help us to show the efficiency and the stability of this proposed estimators. 
We will see that the numerical integration error may also reduce this time-delay 
for a special class of functions. 

2 Derivative Estimations by Using Jacobi Orthogonal 
Series 

Let f s = f + w be a noisy function defined on an open interval I C M, where 

/ G C n (I) with 7i G N and w be a noise[l which is bounded and integrable with 

a noise level 5, i.e. S = sup |n7(x)|. Contrary to |19j where Legendre polynomials 
xei 

were used, we propose to use, as in |14ll5j . truncated Jacobi orthogonal series 
so as to estimate the n th order derivative of /. In this section, we are going to 
give a family of causal Jacobi estimators by using Jacobi polynomials defined 
on [0,1]. From now on, we assume that the parameter h > and we denote 
Ih '■= {x € I; [x — h, x] C /}. 

The n th order Jacobi polynomials (see [H]) defined on [0,1] are defined as 
follows 

p^Ht) = ±( n+ j a )(l + J j )(t-iy i ->(ty (i) 

where a, ft G] — 1, +oo[. Let g\ and g 2 be two functions which belong to C([0, 1]), 
then we define the scalar product of these functions by 

(3i(0,52(-)) Q b : = / w a ^(t) gi {t)g 2 {t)dt, 
Jo 



5 More generally, the noise is a stochastic process, which is bounded with certain 
probability and integrable in the sense of convergence in mean square (see [lip. 
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where w a ^(t) = (1 — t) a t^ is a weighted function. Hence, we can denote its 
associated norm by || • \\ a ,p- We then have 



,|p(^)||2 1 r(a + n + l)r(f5 + n+l) 

11 " 2n + a + /3 + ir(a + (3 + n+l)r(n+l)' 



Let us recall two useful formulae (see [3T|) 

p( aJ) (t)w a ,p{t) = — — [w a+n ^ +n {t)\ (the Rodrigues formula), (3) 
i[Pt P \t)\ = (n + a + l 3 + l)PtX^ +1 \t). (4) 



Let us ignore the noise w for a moment. Since / is assumed to belong to 
C n (I), we define the q th (q G N) order truncated Jacobi orthogonal series of 
f ljl \x — ht) (t G [0, 1]) by the following operator: Va; G In, 



i a+n ' p+n) (-),f^(x-h-)) 



n(") tin. +h\ _ / a+n,f)+n p (a+n,p+n) , .. 

U h, a ^, q I\ x m ) — 2^ ||p(a+rc.,/3+n)|| 2 1 

i=0 \\a+n,P+n 

(5) 

We also define the (q + n) th order truncated Jacobi orthogonal series of f{x — ht) 
(t G [0, 1]) by the following operator 

1+n(P^\-),f(x-h-)) 

V* S I h , D^J(x - th) := £ ^ P^\t). (6) 

i=0 \\a,P 

It is easy to show that for each fixed value x, D^ a ^ g f(x — h-) is a poly- 
nomial which approximates the function f(x — h-). We can see in the following 
lemma that D^ a ^ f(x — h-) is in fact connected to the n th order derivative of 
Dft a p f(x — h-). It can be expressed as an integral of /. 

Lemma 1. Let / 6 C n (I), then we have 



V- e h, D^ q f(x -th) = j-^^ 



Moreover, we have 



Vx G 



h, Di n l p q f(x - th) = J Q a ,p, n . q .t(r)f(x - hr)dT, (8) 



where Q a ,p,n,q,t(r) = w<x,p(r) ^ C ol ^^ i P> a+n,P+n ' {t)P£U \t), and 
C affnt = ^^V^^^ ^h «,/?G]-l,+oo[. 
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Proof. By applying n times derivations to © and by using (jj), we obtain 



dF 

=£ 

i=0 
9 



(0) 



h, a ,e, q f( x - th ) 



||p(",/3)||2 



iftf J (*) 



(9) 



i=0 



||p(",/3)||2 



r(a + /3 + n + i + 1) 



By applying two times the Rodrigues formula given in §3§ and by taking n 
integrations by parts, we get 



l a+n ^ +n) {-),f^(x-h-)) 



, a+n , 0+n {T)Pt +n ' P+n \r) /<">(* - hr)di 



1{ -^wf +n+ ^ +n+l {T)f^\x-hr)d, 



i-\-ri 



(~h) n Jo i! 



1 



1 (n + i)! 



(n+i) 



o, /9 (r)Pi5f ) (r)/(x-/ir)dr. 



(r) /(a: - /ir)dr 



Then, after some calculations by using ((2]) we can obtain 
/p.(«+^+»)(.) ! ;(n)( :r _ /l .)\ 

\ / C 



I -P. 



{a+n,P+n) 112 



(«./3) 



i-\-n 



(■),f(x-h-)) 



a + n . , p + n 

P(a + /3 + 2 ?l + i + 1) 



Finally, by taking and © we obtain 



V^e4 D^Mx-th) 



(-h) n dt n 



D 



(0) 

h,a,l3,q 



f(x - th) 



The proof is complete. 



(10) 



(11) 



□ 



If we consider the noisy function f s , then it is sufficient to replace f(x — h-) 
in © by f(x-h-). In [H], for a given value t T g [0, 1], D^ afjq f{x-t T h) (with 
a, /3 € N and 5 < a + n) was proposed as a point-wise estimate of f^ n \x) by 
admitting a time-delay t T h. We assume here that these values a and (3 belong 
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to ]— 1, +oo[. This is possible due to the definition of the Jacobi polynomials. 
Contrary to |14j , we do not have constraints on the value of the truncation order 
q. Moreover, the function Q a ,/3,n,q.t is easier to calculate than the one given in 
|12j . Thus, we can define the extended point-wise estimators as follows. 

Definition 1. Let f d = f + uj be a noisy function, where f 6 C n (I) and w be a 
bounded and integrable noise with a noise level 5. Then a family of causal Jacobi 
estimators of f( n ) is defined as 



Vse/ fc) D%l t0 J s (x-t T h) 



a,P,n,q,t T (u)f (x - hu)du, (12) 



(~h) n Jo 

where a, G] — 1, +00 [, q, n € N and t T is a fixed value on [0, 1]. 

Hence, the estimation error comes from two sources : the remainder terms in 
the Jacobi series expansion of - h-) and the noise part. In the following 

proposition, we study these estimation errors. 

Proposition 1. Let f s be a noisy function where f € C n+1+q (J) and zu be 
a bounded and integrable noise with a noise level 8. Assume that there exists 
M n+ i +q > such that for any x <E I, |/^ n+9+1 ^(a:)| < M n+ i +q , then 



D hlf3, q f 5 (x - - f {n \x - t T h) < C q , U h" +1 + E q , t —, 



(13) 



where C, 



q,t T 



M, 



"9 ( (n+l+ q y. Jo \ un+1+q Q a ,l3,n,q,t T ( u )\ du + (^FT)t) md 



E q ,t T = Jq 1 \Qa,p,n,q,u( u )\ du. Moreover, if we choose h = 



the 



have 



(9+l)C,,t T 



(14) 



Proof. By taking the Taylor series expansion of / at x, we then have for any 
x € In that there exists £ E]x — h, x[ such that 



I _l 1 \n+l+qj-n+l+q 
f(x - trh) = f n+q (x trh) + { / T & +1+q) (0, 

[n + 1 + qy. 



(15) 



n+q (—hVP 

where f n+q {x — t T h) = — -f^\x) is the (n + q) th order truncated Taylor 



3=0 



series expansion of f(x — t T h). By using ((SJ) with f n+q (x — h-) we obtain 



fn+q( x t T h) — 

Thus, by using (fl"2")l and ([T^]) we obtain 

D^J^-trh)-f^ q (x-t T h) 

{-hy +1 



0,n,q,u(r)f n +q(x - hr)dr. 



(16) 



(n + l + q)\J 
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Consequently, if for any x E I \f^ n+1+9 '(x)\ < M n+ i +q , then by taking the q 
order truncated Taylor series expansion of f^ n \x — t T h) 

/(->(* - t T h) = fi%( X t rh ) + 



th 



we have 

,(n) 



D%l p J(x-t T h)-fM(x-t T h) 



< 



D 



(») 

h,a.,l3,qJ 



f( X - trh) - 0JX -t T h) + fi%( X - t T h) - /<»> (x - t T h) 



<h q+L M n+1+q 



Since 



1 



(n+l + q)\J 



r n+1+q Q a ^ M (r)\ dr + 



(9 + 1)! 



(17) 



= \ D %l,e, q [f 5 (x-tTh)-f(x-t T h)] 

6 

by using (fl~7) we get 

oo 

<, ft /(^-a)-< ft! /(^-a) 

D^J(x-t T h)-f^(x-t T h) 

<C qiU h« +1 +E q!tT -j^, 
where C g ,t T = M n+1+q ( (ft+1+g)! Jo | T ™ +1+9< 3a^,n,«,t T ( t ) | dr + rJ^TJl) and 

S 9,*t = Jo 1 IQa,/3,n,g,t T ( T )l dr - 

Let us denote the error bound by ij){h) = C q ^ T h q+1 +E q j T jpr- Consequently, 



< 



we can calculate its minimum value. It is obtained for h* 
and 

n + 1 + q ( q + 1 



(?+l)C,, tT ° 



<P(h* 



(18) 
□ 



The proof is complete. 

Let us mention that if we set t T = 9 q+ i, the smallest root of the Jacobi 



polynomial P q +i in ([5]), then D)™' a ^ q f(x — 6 q+ ih) becomes the (q + 1) 

order truncated Jacobi orthogonal series of f( n '(x — 8 q+ ih). Hence, we have the 
following corollary. 
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Corollary 1. Let f £ C n+2+q (I) where q is an integer. If we set t T = 9 



smallest root of the Jacobi polynomial pfa-+">^+ n ) j n ^jgj) an d we assume that 
there exists M n+2 + q > such that for any x £ I. |/^™ +9+2 ^(a;)| < M n +2+ q , then 
we have 



the 



D 



h) 



,h q 



+2 



E q,e q+ i hn i 



where C, 



'fi q +l - M n+2+q ((„+g +2 )! / I' 



■n+q+2 



given in Proposition]]] Moreover, if we choose h 
then we have 



E q ,e q+1 is 



Qa,P,n,q,6 t+1 {r)\dT + (f+2)T 
nE 1,<>a + l e 



D 



L1«,,A*-<W>0-/ W (z 



Jq+1 



h) 



(q+2)C q ,e a 



Proof. If t T = q +i, the smallest root of polynomial p(™+ n >P +n > m then we 
have 



D^U q f(x-e q+1 h) 



9+1 < P. 

E 

i=0 



t +n ^ +n \-)J^(x-h-)) 



a+n,f3+n p(a+n,/3+n) t 



I -Pi 



(a+n,/3+n) ■ ■ 2 



?9+lJ 



This proof can be completed by taking the (n + q + 1) order truncated Taylor 
series expansion of / as it was done in Proposition [T] □ 

The numerical calculation of E q ,§ q+1 for q, n £ N and a, (3 £] — 1, 10] shows 
that E q ,g q+1 increases with respect to g. Hence, in order to reduce the noise 
influence it is preferable to choose q as small as possible. However, C q ,g q+1 de- 
creases with respect to q. A compromise consists in choosing q = 2. If we take 
d!"^ - 2 / <5 ( a; — ^2^) as an estimator of f( n \x), then we produce a time-delay 



6 2 h. For this choice of 6 2 , we have D { ^ a p i f d (x - 9 2 h) = D][ l > ap 2 f A (x - 6 2 h). 

We can see that the estimators D^ a ^ 2 f S ( x ) d° not produce a time-delay but 
Ci,0 2 < C 2t o and Ei_g 2 < E 2i q. This generally introduces more important estima- 
tion errors. Consequently, so as to estimate f(™>(x) we use D^' a n 1 f s (x — 8 2 h) 
which presents a time-delay. 



3 Numerical Experiments 

In order to show the efficiency and the stability of the previously proposed 
estimators, we give some numerical results in this section. 

From now on, we assume that f (xi) = f(xi) + cw(x,i) with x, = T s i for i = 
0, • • ■ , 500 (T s = j^q), is a noisy measurement of f(x) — exp(^-|) sin(6x+7r). The 
noise cw(xi) is simulated from a zero-mean white Gaussian iid sequence by using 
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the Matlab function 'randn' with STATE reset to 0. Coefficient c is adjusted in 
such a way that the signal-to-noisc ratio SNR = 101og 10 ( ^\l^(t)\ 2 ^j * s cc l ua l 
to SNR = 22.2dB (see, e.g., [7] for this well known concept in signal processing). 
By using the well known three-sigma rule, we can assume that the noise level for 
czu is equal to 3c. We can see the noisy signal in Figure [T] We use the trapezoidal 
method in order to approximate the integrals in our estimators where we use 
m + 1 discrete values. The estimated derivatives of / at Xi € I = [0, 5] are 
calculated from the noise data f s (xj) with ] where h = mT s . 



f (x.)=exp(-x./1.2) sin{6 x.+7t)+0.5ffi(x.) 



0.6 r 
0.4 
0.2 
■ 
-0.2 
-0.4 
-0.6 
-0.8 ■ 



i \ 



1 / 



Fig. 1. Signal and noisy signal. 



We can see the estimation results for the first order derivative of / in Figure[2j 
The corresponding estimation errors are given in Figure[3]and in FigureE] We can 
see that the estimate given by the causal Jacobi estimator with integer parame- 
ters introduced in [T5] (dash line), produces a time-delay of value 9 q +\h = 0.11. 
The estimate given by the causal Jacobi estimator with extended parameters 
(dotted line) is time-delay free. Firstly, the root values 9 q +i for q = 0, 1 can 
be reduced with the extended negative parameters, so does the time-delay. Sec- 
ondly, the numerical integration method with a negative value for /3 produces a 
numerical error which allows us to finally compensate this reduced time-delay. 
This last phenomena is due to the fact that the initial function / satisfies the 
following differential equation + kf = g where feel and g is a continuous 
function. Consequently, in the case of the first order derivative estimations, we 
can verify that this numerical error which depends on / may reduce the effect 
of the error due to the truncation in the Jacobi series expansion. This is due to 
the fact that the truncation error depends on /' 2 '. Hence, the final total error is 
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O(g). Finally, since D^ T ^ a ^ q f 5 (xi — 6 2 h) produces a time-delay of value f3 2 /i, 
we give in Figure [5] the errors D^ Ts a /3 q f s (xi - 9 2 h) - - 9 2 h). 



Estimations 

6r 




1 2 3 4 5 

x 

Fig. 2. Estimations by Jacobi estimators D^ a p q f S (xi) with h = 0.2, j3 = —0.25, a = 
l,g = and D^ agq f( Xi - 8 2 h) with h = 0.4, a = /3 = 0, q = 1, 9 2 = 0.276. 



Estimations errors 
0.25 r , 




1 2 3 4 5 



Fig. 3. D^ a0 J s (xi) - with h = 0.2,/? = -0.25, a = l,q = 0. 
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Estimations errors 

2r 




1 2 3 4 5 



Fig. 4. D^ J 5 {xi -6 2 h)- f m ( Xi ) with h = 0.4, a = /3 = 0, q = 1, 9i = 0.276. 



Estimations errors 

0.25 r i 




-0.2 1 1 1 1 > 

1 2 3 4 5 

x 



Fig. 5. D^ a q f s (xi - 6 2 h) - ( Xi - 6 2 h) with h = 0.4, a = /3 = 0, q = 1, 6 2 = 0.276. 
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